Add option to control MAC address reuse for VR public NICs#13001
Open
bernardodemarco wants to merge 1 commit intoapache:mainfrom
Open
Add option to control MAC address reuse for VR public NICs#13001bernardodemarco wants to merge 1 commit intoapache:mainfrom
bernardodemarco wants to merge 1 commit intoapache:mainfrom
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## main #13001 +/- ##
=============================================
- Coverage 18.08% 3.52% -14.56%
=============================================
Files 6037 464 -5573
Lines 542437 40151 -502286
Branches 66420 7557 -58863
=============================================
- Hits 98088 1415 -96673
+ Misses 433333 38548 -394785
+ Partials 11016 188 -10828
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Member
Author
|
@blueorangutan package |
|
@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 17451 |
bernardodemarco
force-pushed
the
change-mac-address-of-public-nics-of-vrs
branch
from
82d70a0 to
0cd6426
Compare
Member
Author
|
@blueorangutan package |
|
@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 17452 |
Member
|
@bernardodemarco
If the public ip services are down for only 1 second or so, I think this PR is good. |
bernardodemarco
force-pushed
the
change-mac-address-of-public-nics-of-vrs
branch
from
0cd6426 to
1144978
Compare
Member
Author
|
Hello, @weizhouapache
Sure! Below are described some connectivity stress tests that I performed in my local environment. They were executed over a VPC (CIDR 10.0.0.0/16), with a tier (CIDR 10.0.0.0/24) whose ACL was configured to allow all ingress and egress network traffic. I acquired a public IP (192.168.122.15) for the VPC and assigned it to a VM provisioned in the tier. In the VM, I configured a UDP server with Node.js and started it as a daemon: UDP Serverconst udp = require('dgram')
const fs = require('fs')
const server = udp.createSocket('udp4')
const port = 2222
const logStream = fs.createWriteStream('udp.log', { flags: 'a' })
server.on('error', (error) => {
logStream.write('Error: ' + error)
server.close()
})
server.on('message', (msg, info) => {
const message = msg.toString()
const logEntry = `[${new Date().toISOString()}] ${info.address}:${info.port} - ${message}\n`
logStream.write(logEntry)
console.log(logEntry.trim())
})
server.on('listening', () => {
logStream.write('Server is listening at port ' + port)
})
server.on('close', () => {
logStream.write('Socket is closed')
logStream.end()
})
server.bind(port)Next, I developed a client script to send UDP datagrams to the server every 100 milliseconds: UDP Clientconst udp = require('dgram')
const host = '192.168.122.15'
const port = 2222
const sleepInterval = 100
const client = udp.createSocket('udp4')
function randomIntFromInterval(min, max) {
return Math.floor(Math.random() * (max - min + 1) + min)
}
let i = 0
async function sendDataToServer() {
const message = 'load test #' + (i++).toString() + ' -- data:' + Date.now().toString(36).repeat(randomIntFromInterval(100, 150))
console.log(message)
const data = Buffer.from(message)
client.send(data, port, host, (error) => {
console.log('sending data to the server')
if (error) {
console.log('error - closing socket')
client.close()
}
})
}
setInterval(sendDataToServer, sleepInterval)Non-redundant VPC cleanupReusing the same MAC address
LogsNetwork traffic obtained from
|
Member
Author
|
@blueorangutan package |
|
@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17471 |
Member
thanks @bernardodemarco can you grep "arping" in the /var/log/cloud.log in the backup VR (now it should be primary VR) ? |
Member
Author
|
Hello, @weizhouapache
Sure, here are the
|
Member
|
@bernardodemarco |
Contributor
|
@bernardodemarco , all looks good. Do you have some 3rd person testing this? |
Member
Author
|
Hello, @DaanHoogland
Not actually. Maybe @erikbocks or @GaOrtiga could help testing this one? Let's see |
|
[SF] Trillian Build Failed (tid-15917) |
|
[SF] Trillian test result (tid-15916)
|
|
[SF] Trillian test result (tid-15919)
|
bernardodemarco
force-pushed
the
change-mac-address-of-public-nics-of-vrs
branch
from
1144978 to
b0bb69e
Compare
Member
Author
|
I have rebased the PR's branch with |
Member
Author
|
@blueorangutan package |
|
@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17576 |
Member
Author
|
@DaanHoogland @weizhouapache can we trigger the CI again for this one, please? |
Member
|
@blueorangutan test matrix |
|
@weizhouapache a [SL] Trillian-Jenkins matrix job (EL8 mgmt + EL8 KVM, Ubuntu22 mgmt + Ubuntu22 KVM, EL8 mgmt + VMware 7.0u3, EL9 mgmt + XCP-ng 8.2 ) has been kicked to run smoke tests |
|
[SF] Trillian Build Failed (tid-15936) |
|
[SF] Trillian test result (tid-15935)
|
|
[SF] Trillian test result (tid-15938)
|
|
[SF] Trillian test result (tid-15937)
|
Member
|
@weizhouapache could we rerun the tests for xcpng82 and vmware-70u3 to see if it matches #12198 and confirm whether the failures are due to the environment? |
Member
|
@blueorangutan test ol9 xcpng82 |
Contributor
|
@blueorangutan test ol9 xcpng82 |
|
@DaanHoogland a [SL] Trillian-Jenkins test job (ol9 mgmt + xcpng82) has been kicked to run smoke tests |
Contributor
|
@blueorangutan test ol8 vmware-70u3 |
|
[SF] Trillian Build Failed (tid-15970) |
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
|
[SF] Trillian test result (tid-15975)
|
bernardodemarco
force-pushed
the
change-mac-address-of-public-nics-of-vrs
branch
from
b0bb69e to
a1a4e8b
Compare
Member
Author
|
@blueorangutan package |
|
@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ACS currently uses the same MAC address for the public NICs of VRs when implementing redundant VRs or when restarting an isolated network or VPC with cleanup. In the case of restarts with cleanup, ACS initializes a new VR with its NICs and subsequently applies the corresponding configuration and IP addresses.
In this scenario, when VRs are cleaned up while there is intensive traffic on the public network, traffic may still be directed to the same MAC address as the previous VR before the new VR’s public NIC is fully configured. As a result, the VR’s operating system may interpret this traffic as a spoofing attack and begin dropping packets. To address these cases, the MAC address of the public NIC can be changed when VRs are recreated during cleanup. This ensures that network traffic is only directed to the new VR once it is fully configured.
To support this behavior in ACS, the
keepmacaddressonpublicnicparameter has been added to thecreateNetwork,createVPC,updateNetwork, andupdateVPCAPIs. When set to true (the default, preserving existing behavior), ACS reuses the same MAC address for the public NIC across VRs during redundancy or recreation. When set to false, newly created public NICs are always assigned a new MAC address.Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Screenshots (if appropriate):
How Has This Been Tested?
Authorization validation
Adminaccounts have permission to manipulate thekeepmacaddressonpublicnicparameter in the following APIs:createNetwork;updateNetwork;createVPC;updateVPC.Network connectivity tests
See #13001 (comment)
Behavior validation
Isolated networks
keepmacaddressonpublicnicparameter set totrue1e:01:89:00:00:0efalseand performed a network clean up1e:01:ee:00:00:0eand1e:01:26:00:00:0ekeepmacaddressonpublicnicparameter set tofalse1e:01:73:00:00:0d1e:01:32:00:00:0dand1e:01:82:00:00:0dRedundant isolated networks
keepmacaddressonpublicnicparameter set totruekeepmacaddressonpublicnicparameter set tofalse.VPCs
keepmacaddressonpublicnicparameter set totrue1e:01:3a:00:00:0dfalseand performed a VPC clean upRedundant VPCs
keepmacaddressonpublicnicparameter set totruekeepmacaddressonpublicnicparameter set tofalse